In this paper, we propose a method for the approximation of the solution of
high-dimensional weakly coercive problems formulated in tensor spaces using low-rank
approximation formats. The method can be seen as a perturbation of a minimal residual
method with a measure of the residual corresponding to the error in a specified solution
norm. The residual norm can be designed such that the resulting low-rank approximations
are optimal with respect to particular norms of interest, thus allowing to take into
account a particular objective in the definition of reduced order approximations of
high-dimensional problems. We introduce and analyze an iterative algorithm that is able to
provide an approximation of the optimal approximation of the solution in a given low-rank
subset, without any a priori information on this solution. We also
introduce a weak greedy algorithm which uses this perturbed minimal residual method for
the computation of successive greedy corrections in small tensor subsets. We prove its
convergence under some conditions on the parameters of the algorithm. The proposed
numerical method is applied to the solution of a stochastic partial differential equation
which is discretized using standard Galerkin methods in tensor product spaces.